Search for: All records

Data center cooling systems have undergone a major transformation in the persistent pursuit of better performance and lower energy use. Liquid cooling systems, particularly direct-to-chip systems, have emerged as a promising solution to address the increasing heat dissipation challenges. One critical component of such systems is the filtration mechanism, responsible for safeguarding the integrity and efficiency of the cooling process. These factors are pivotal in ensuring the reliable and sustainable operation of liquid cooling systems in high-demand applications, where electronic components continually push the boundaries of heat generation. This study undertakes a thorough examination of filters of different mesh size used in direct-to-chip liquid cooling systems. The research is multifaceted, encompassing the evaluation of filter performance, pressure drop characteristics, and long-term durability. The methodology employed in this research combines testing with a coolant distribution unit and rack setup to provide a holistic perspective on filter functionality. Findings from this study shed light on the key parameters that influence filter performance. Ultimately, the results of this research promise to contribute significantly to the advancement of direct-to-chip liquid cooling systems, facilitating the continued evolution of electronics in diverse fields, such as high-performance computing, data centers, and emerging technologies. With a focus on enhancing system reliability, efficiency, and sustainability, this study seeks to provide a valuable resource for engineers and researchers in the pursuit of effective cooling solutions for cutting-edge electronic applications. 

Effective cooling is crucial for high-power liquid-cooled servers to ensure optimal performance and reliability ofcomponents. Thermal characterization is necessary to ensure that the cooling system functions as intended, is energy efficient, and minimizes downtime. In this study, a proposed methodology for thermal characterization of a high-powerliquid-cooled server/TTV [server and TTVs (thermal test vehicle) are used interchangeably] is presented. The server layout includes multiple thermal test vehicle setups equipped with direct-to-chip cold plates, with two or more connected in series to form a TTV cooling loop. These cooling loops are connected in parallel to the supply and return plenums of the cooling loop manifold, which includes a chassis-level flow distribution manifold. To obtain accurate measurements, two identical server/TTV prototypes are instrumented with sensors for coolant flow rate and temperature measurements for every TTV cooling loop. Four ultrasonic flow sensors are installed in the flow verification server/TTV to measure the coolant flow rate to each TTV cooling loop. In the thermal verification server, thermistors are installed at the outlet of each GPU heater of TTV cooling loop to log temperature measurements. The amount of heat captured by the coolant in each TTV cooling loop is subsequently estimated based on the flow rates determined from the flow verification server.This methodology enables precise characterization of the thermal performance of high-power liquid-cooled servers,ensuring optimal functionality, energy efficiency, and minimized downtime. 

Abstract Direct Liquid Cooling (DLC) has emerged as a promising technology for thermal management of high-performance computing servers, enabling efficient heat dissipation and reliable operation. Thermal performance is governed by several factors, including the coolant physical properties and flow parameters such as coolant inlet temperature and flow rate. The design and development of the coolant distribution manifold to the Information Technology Equipment (ITE) can significantly impact the overall performance of the computing system. This paper aims to investigate the hydraulic characterization and design validation of a rack-level coolant distribution manifold or rack manifold. To achieve this goal, a custom-built high power-density liquid-cooled ITE rack was assembled, and various cooling loops were plugged into the rack manifold to validate its thermal performance. The rack manifold is responsible for distributing the coolant to each of these cooling loops, which is pumped by a CDU (Coolant Distribution Unit). In this study, pressure drop characteristics of the rack manifold were obtained for flow rates that effectively dissipate the heat loads from the ITE. The pressure drop is a critical parameter in the design of the coolant distribution manifold since it influences the flow rate and ultimately the thermal performance of the system. By measuring the pressure drop at various flow rates, the researchers can accurately determine the optimum flow rate for efficient heat dissipation. Furthermore, 1D flow network and CFD models of the rack-level coolant loop, including the rack manifold, were developed, and validated against experimental test data. The validated models provide a useful tool for the design of facility-level modeling of a liquid-cooled data center. The CFD models enable the researchers to simulate the fluid flow and heat transfer within the cooling system accurately. These models can help to design the coolant distribution manifold at facility level. The results of this study demonstrate the importance of the design and development of the coolant distribution manifold in the thermal performance of a liquid-cooled data center. The study also highlights the usefulness of 1D flow network and CFD models for designing and validating liquid-cooled data center cooling systems. In conclusion, the hydraulic characterization and design validation of a rack-level coolant distribution manifold is critical in achieving efficient thermal management of high-performance computing servers. This study presents a comprehensive approach for hydraulic characterization of the coolant distribution manifold, which can significantly impact the overall thermal performance and reliability of the system. The validated models also provide a useful tool for the design of facility-level modeling of a liquid-cooled data center. 

Abstract Due to the increasing computational demand driven by artificial intelligence, machine learning, and the Internet of Things (IoT), there has been an unprecedented growth in transistor density for high-end CPUs and GPUs. This growth has resulted in high thermal dissipation power (TDP) and high heat flux, necessitating the adoption of advanced cooling technologies to minimize thermal resistance and optimize cooling efficiency. Among these technologies, direct-to-chip cold plate-based liquid cooling has emerged as a preferred choice in electronics cooling due to its efficiency and cost-effectiveness. In this context, different types of single-phase liquid coolants, such as propylene glycol (PG), ethylene glycol (EG), DI water, treated water, and nanofluids, have been utilized in the market. These coolants, manufactured by different companies, incorporate various inhibitors and chemicals to enhance long-term performance, prevent biogrowth, and provide corrosion resistance. However, the additives used in these coolants can impact their thermal performance, even when the base coolant is the same. This paper aims to compare these coolant types and evaluate the performance of the same coolant from different vendors. The selection of coolants in this study is based on their performance, compatibility with wetted materials, reliability during extended operation, and environmental impact, following the guidelines set by ASHRAE. To conduct the experiments, a single cold plate-based benchtop setup was constructed, utilizing a thermal test vehicle (TTV), pump, reservoir, flow sensor, pressure sensors, thermocouple, data acquisition units, and heat exchanger. Each coolant was tested using a dedicated cold plate, and thorough cleaning procedures were carried out before each experiment. The experiments were conducted under consistent boundary conditions, with a TTV power of 1000 watts and varying coolant flow rates (ranging from 0.5 lpm to 2 lpm) and supply coolant temperatures (17°C, 25°C, 35°C, and 45°C), simulating warm water cooling. The thermal resistance (Rth) versus flow rate and pressure drop (ΔP) versus flow rate graphs were obtained for each coolant, and the impact of different supply coolant temperatures on pressure drop was characterized. The data collected from this study will be utilized to calculate the Total Cost of Ownership (TCO) in future research, providing insights into the impact of coolant selection at the data center level. There is limited research available on the reliability used in direct-to-chip liquid cooling, and there is currently no standardized methodology for testing their reliability. This study aims to fill this gap by focusing on the reliability of coolants, specifically propylene glycols at concentrations of 25%. To analyze the effectiveness of corrosion inhibitors in these coolants, ASTM standard D1384 apparatus, typically used for testing engine coolant corrosion inhibitors on metal samples in controlled laboratory settings, was employed. The setup involved immersing samples of wetted materials (copper, solder coated brass, brass, steel, cast iron, and cast aluminum) in separate jars containing inhibited propylene glycol solutions from different vendors. This test will determine the reliability difference between the same inhibited solutions from different vendors. 

In typical data centers, the servers and IT equipment are cooled by air and almost half of total IT power is dedicated to cooling. Hybrid cooling is a combined cooling technology with both air and water, where the main heat generating components are cooled by water or water-based coolants and rest of the components are cooled by air supplied by CRAC or CRAH. Retrofitting the air-cooled servers with cold plates and pumps has the advantage over thermal management of CPUs and other high heat generating components. In a typical 1U server, the CPUs were retrofitted with cold plates and the server tested with raised coolant inlet conditions. The study showed the server can operate with maximum utilization for CPUs, DIMMs, and PCH for inlet coolant temperature from 25–45 °C following the ASHRAE guidelines. The server was also tested for failure scenarios of the pumps and fans with reducing numbers of fans and pumps. To reduce cooling power consumption at the facility level and increase air-side economizer hours, the hybrid cooled server can be operated at raised inlet air temperatures. The trade-off in energy savings at the facility level due to raising the inlet air temperatures versus the possible increase in server fan power and component temperatures is investigated. A detailed CFD analysis with a minimum number of server fans can provide a way to find an operating range of inlet air temperature for a hybrid cooled server. Changes in the model are carried out in 6SigmaET for an individual server and compared to the experimental data to validate the model. The results from this study can be helpful in determining the room level operating set points for data centers housing hybrid cooled server racks. 

Modern day data centers are operated at high power for increased power density, maintenance, and cooling which covers almost 2 percent (70 billion kilowatt-hours) of the total energy consumption in the US. IT components and cooling system occupy the major portion of this energy consumption. Although data centers are designed to perform efficiently, cooling the high-density components is still a challenge. So, alternative methods to improve the cooling efficiency has become the drive to reduce the cooling cost. As liquid cooling is more efficient for high specific heat capacity, density, and thermal conductivity, hybrid cooling can offer the advantage of liquid cooling of high heat generating components in the traditional air-cooled servers. In this experiment, a 1U server is equipped with cold plate to cool the CPUs while the rest of the components are cooled by fans. In this study, predictive fan and pump failure analysis are performed which also helps to explore the options for redundancy and to reduce the cooling cost by improving cooling efficiency. Redundancy requires the knowledge of planned and unplanned system failures. As the main heat generating components are cooled by liquid, warm water cooling can be employed to observe the effects of raised inlet conditions in a hybrid cooled server with failure scenarios. The ASHRAE guidance class W4 for liquid cooling is chosen for our experiment to operate in a range from 25°C – 45°C. The experiments are conducted separately for the pump and fan failure scenarios. Computational load of idle, 10%, 30%, 50%, 70% and 98% are applied while powering only one pump and the miniature dry cooler fans are controlled externally to maintain constant inlet temperature of the coolant. As the rest of components such as DIMMs & PCH are cooled by air, maximum utilization for memory is applied while reducing the number fans in each case for fan failure scenario. The components temperatures and power consumption are recorded in each case for performance analysis